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ABSTRACT 


This thesis deals with a type of stochastic optimization problem where the decision 
maker does not have complete information concerning the objective function. Specif- 
ically, we consider a discrete time-and-space search optimization problem where we 
seek to find a moving target in an area of operations. There are two sources of uncer- 
tainty: the target location and the sensor performance. We formulate the objective 
function for this problem in terms of a risk measure of a parameterized random vari- 
able and consider three cases involving various degrees of knowledge about the sensor 
performance. In all cases, we consider both the expectation and superquantile risk 
measures. While the expectation results in an objective function representing the 
probability of missing the target, the superquantile gives rise to more conservative 
search plans that perform reasonably well even under exceptional circumstances. In 
the case of incomplete information about the distribution of the sensor performance, 
we approximate the random variable using a nonstandard regression that minimizes 
the error induced in some sense. We examine the cases in a series of numerical 


examples. 
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EXECUTIVE SUMMARY 


In this thesis, we discuss a type of stochastic optimization problem where the de- 
cision maker does not have complete information concerning the objective function. 
Specifically, we consider a discrete time-and-space search optimization problem (SP) 
of routing search assets in order to detect a moving target, such as a drug smuggler, 
in an area of operations. 

We assume that each searcher is equipped with one imperfect sensor, which 
is not subject to false-positive detections. The searchers are all alike, equipped with 
the same type of sensor, and they are allowed to divide their effort across multiple 
cells in arbitrarily small portions. The goal is to determine a search plan such that 
the probability of missing the target is minimized, without having the full knowledge 
of the two sources of uncertainty: the target location and the sensor performance. 
We assume that the target paths are probabilistically known via intelligence reports; 
for example from data acquired by AIS (Automatic Identification System). We use 
a Markov chain model to generate the paths with three stationary probabilities. We 
assume that the sensor performance depends on environmental conditions, and we 
use visibility to represent these conditions. 

We formulate the objective function of SP in terms of a risk measure of a 
parameterized random variable and consider three cases involving various degrees of 
knowledge about the sensor performance. In all cases, we consider both the expecta- 
tion and superquantile risk measures. While the expectation results in an objective 
function representing the probability of missing the target, the superquantile gives 
rise to more conservative search plans that perform reasonably well even under ex- 
ceptional circumstances. 

In the first two cases, we consider a random detection rate for the sensor per- 
formance with known probability distribution. In the third case, the distribution of 


the sensor performance is unknown and we approximate the corresponding random 


XV 


variables by a linear combination of well-known factors using a nonstandard regres- 
sion. Based on a table of observed sensor performances, that regression minimizes 
the error induced by the approximation in some sense. We compare the obtained 
regression coefficients with the ones resulting from a least-squares linear regression 
model. Using the approximations in the objective function of SP, we obtain an ap- 
proximate problem that we solve for situations involving stochastic information about 
the factors in the regression model. 

We examine the cases in a series of numerical examples. In all cases, the 
numerical results show that the plans obtained using superquantile as the risk measure 
spread the searchers over the area of operations, covering a larger area with smaller 
searcher fractions than when using expectation. These plans handle exceptional target 
paths more effectively. If the goal is to make sure that the probability of missing the 
target does not exceed a relatively high threshold, then the decision maker should 
rely on the superquantile at probability level a as the risk measure. One example 
shows that if the goal is to avoid probabilities of missing the target of 70% or higher, 
then the decision maker should use superquantile with a = 0.90. In that situation 
there is a 92% chance of obtaining better mission outcomes than the goal, while there 
is only 81% chance of getting better than the goal when using the expectation. This 
difference might be critical when choosing a search plan and it even is more significant 


for faster moving targets. 


Xvl 


I. INTRODUCTION 


A. MOTIVATION AND BACKGROUND 

Stochastic optimization problems arise in a diverse spectrum of real-life sit- 
uations, but they can be challenging to formulate and solve since they involve un- 
certainty and unknown parameters (see for example Shapiro et al., 2009; Wallace & 
Ziemba, 2005). 

One application area is engineering design, where the goal is to minimize the 
cost of a design, which might involve a simple objective function and complicated 
chance constraints. For example, the design must be able to resist future unknown 
loads, material properties, and environmental conditions; see Arora and Wang (2005), 
Royset et al. (2006), and Rockafellar and Royset (2010) for formulations and Luedtke 
and Ahmed(2008), and Shapiro et al. (2009) for algorithms and properties of these 
constraints. Another area of application is financial engineering where studies show 
the importance of selecting a suitable risk measure (Artzner et al., 1999) in portfolio 
management. A particular risk measure is the superquantile (also called conditional 
value-at-risk), which is coherent (Rockafellar & Uryasev, 2002), and therefore may be 
suitable for use in many applications. 

In military operations, it is common for leaders to make decisions without 
the full knowledge of future events such as enemy actions, environmental conditions, 
and asset availability. We refer to a combination of future events as a scenario. Due 
to advanced coordination and planning that may involve numerous operational and 
logistical units, leaders may have to ignore the possibility of changing their decision 
in the middle of the action. Hence, recourse may not be available after a scenario is 
revealed. 

In this thesis, we focus on search for a target in an area of operations where 
uncertainty is prevalent. Specifically, we consider two types of uncertainty: sensor 


performance and target location. The sensor performance depends on several aspects, 


e.g., operator experience, environmental conditions such as visibility, and target char- 
acteristics. Search problems of this kind arise in military operations as well as search 
and rescue operations. 

While some stochastic problems involve simple objective functions and com- 
plicated chance constraints, our search problem has an objective function that is more 
complex since it is defined by a risk measure of a parametrized random variable. A 
risk measure maps a random variable to the extended real numbers. We focus on the 
superquantile as the risk measure and compare the resulting search plans with the 
ones obtained using the expectation. We are particularly interested in cases where 
the probability distribution of the parametrized random variable is unknown, but a 


table of realizations is available. 


B. CONTRIBUTIONS 

This thesis is the first to consider superquantile as a risk measure in the con- 
text of search planning. Existing approaches focus exclusively on expectation. Since 
the superquantile is averse as explained below, it allows the analyst to plan for rare 
and undesirable events. We also consider search problems with incomplete informa- 
tion about sensor performance and we construct function approximations based on 


nonstandard regression models for use in subsequent optimization problems. 


C. THESIS ORGANIZATION 

Chapter II defines the search problem we are interested in and formulates a 
model that will be used for theoretical and numerical studies. The chapter describes 
three distinct cases and scenarios defined by the sensor detection rates and target 
paths used in numerical studies. 

Chapter III presents two of the cases where the probability distribution of 
the detection rate is known. We solve and compare these cases for expectation and 


superquantile risk measures. 


Chapter IV discusses the third case, where the probability distribution of the 
detection rate is unknown and the decision maker only has a table of realizations of 
detection rates, collected for a certain time period in the past, for a certain visibil- 
ity condition. We use a nonstandard regression model, and the obtained regression 
coefficients are then used in the search problem as an approximation. 

Chapter V summarizes the theoretical and numerical results, and recommends 


further research. 
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II. MODEL FORMULATION AND RISK 
MEASURE 


This chapter describes a search problem whose solutions we compare in the 


following chapters under various assumptions. 


A. PROBLEM DEFINITION 

In this thesis, we study a search-optimization problem (SP) where we consider 
an Area Of Interest (AOI) defined by a finite set of cells C = {1,2,...,C}, and we let 
time also be discrete, described as a finite set of time periods Jo = {0} U7, where 
te eee a 

We want to route search assets in this AOI in order to detect a target such as 
a drug smuggler who is trying to cross it. The AOI includes a given number of bases, 
located in specific cells, with different logistical capabilities. Each one of these bases 
is able to accommodate a given number of search assets. 

Based on Royset and Sato (2010), we consider a single target, but an arbitrary 
number of searchers. Both target and searchers have their own initial positions in the 
AOI. The initial positions for the searchers correspond to bases they are allocated 
to at time 0 or other cells in the AOI representing some area from which they start 
their missions, accounting for situations where some assets might have already been 
allocated for another low priority mission. We assume that the searchers divide their 
effort across multiple cells in arbitrarily small portions. This assumption allows for 
continuous variables in the model formulation. Many of the results in this thesis, 
however, generalize straightforwardly to the integer case. 

At each time period t € 7, searchers and target either occupy a cell c or are 
transiting between cells. When in cell c, they are allowed to move to any cell which 
is adjacent to c as defined by S(c) CC. 


We assume that each searcher is equipped with one imperfect sensor, which 


is not subject to false-positive detections. We also assume that the searchers are all 
alike, equipped with the same type of sensor. At each time period t € 7 during which 
a searcher is in cell c, its sensor takes one glimpse to see if the target is present in 
the cell. We define R,, as the corresponding nonnegative detection rate, in cell ¢ and 
time period t, of the sensor. The increased noise caused by the searcher movements 
between cells affects the sensor performance. Therefore we assume that the sensor is 
inactive during transit between cells. Furthermore its performance is uncertain due 
to unknown environmental conditions and target characteristics. 

In the AOI there is one target whose initial position is obtained from intel- 
ligence reports. The target moves conditionally deterministic and a possible target 
path is denoted by w = (wy, Wa, ...,wr) € QC C7, where u, € C is the target’s cell at 
time period t on path w, with a corresponding probability 6(w) of the target actually 
taking the path w, where 22 is the set of all possible paths. We let P.; be a Bernoulli 
random variable, taking the value 1 if the target is in cell c at time period t. We define 
y(v) as the probability of a particular scenario v occurring, where v is a realization 
of V = (Vereeeter = ((Pot, Ret)" )cecter, a vector representing the random target 
path and random detection rates. 

Our goal is to determine a plan that routes the available search assets over 
the AOI such that the probability of missing the target is minimized, without the 
full knowledge of future environmental conditions that might affect the sensor per- 


formance. 


B. MODEL FORMULATION 
We formulate the search problem as a convex nonlinear program, similar to 
Royset and Sato (2010), but we generalize it by using R(-) to denote a risk measure 


as defined in the following section. The model SP takes the following form: 


Model SP: 


Indices 


Parameters 


Die 


Random Variables 


Pod 


Hed 


cells (c,c) € C = {1,...,C}). 
time periods (t € Jp = {0} UT, 7 = {1,..., T}). 


allowable search moves the searchers can carry out 


starting from cell c. 


number of time periods needed for a searcher to 
move directly from cell c to cell c’ and search c’. 
number of searchers positioned at cell c in 


time period 0. 


1 if target is in cell c in time period t, otherwise 0; 
probability distribution of P., defined by 6(w) 
through Prob(P.; = 1) = Prob(u; = c). 
nonnegative detection rate in cell c at time t. 
vector of random variables V = (V.4)cecteT = 

= ((P.1, Roz)" )eeczer, With realization v = 

= (Vet )cecteT = ((Pets Pet) | )eecteT and 


Prob(V = v) = y(v). 


Decision Variables 


Lec! st 


Function 
F(a, v) 


Formulation 


c'ES(c) 


number of searchers that occupy cell c at time t 


and move to cell c’ next, with x = (tow t)ec'ec,teT- 


probability of missing the target given search plan 


x and realization v of V, where 


F(z,v) = exp {- S- paraatocth (II.1) 


c,c! tEeT 


min R(F(a,V)) (11.2) 
s.t. S- Le ct—-dy . = Ss" Geo, Veee fl (11.3) 
c'€S(c) 
> t= Ve (II.4) 
c' ES (c) 
Lege: <M ee oF (II.5) 


We adopt the same function as in Royset and Sato (2010) for the probability 


of missing the target given a search plan x and a realization v, as seen in (II.1). 


The constraint (II.3) defines the allowable moves the searchers may take and the 


corresponding duration in time periods, avoiding jumps between non-adjacent cells, 


and the constraint (II.4) establishes the searcher initial positions. We discuss the 


numerical results of the model SP for different assumptions about the distribution of 


R.4 and risk measures in Chapters III and IV. 


C. RISK MEASURE 

One approach to SP is to choose a particular scenario v, and then minimize 
F(ax,v) subject to (I11.3)-(II.5). This means that the variability of the future is not 
taken into account and there is no guarantee that the obtained search plan x is 
optimal for F(x,v’), where vu’ 4 v is the observed scenario during the mission. The 
searchers might not be prepared for a scenario that deviates from the one chosen. 
Another alternative is to consider the worst-case scenario and minimize sup,, F(z, v) 
subject to (II.3)-(IL.5). The resulting solution may be highly conservative. Others 
prefer to rely on the expected value of the objective function, which corresponds to 
the probability of missing the target. On average they do well, but the resulting 
search plan might be quite poor in a given scenario. We compare different choices 
of this kind, which correspond to different risk measures 7(-), and see how well the 
resulting search plans perform in some sense. 

As seen in Rockafellar et al. (2008), we say that a risk measure is coherent if 


the following axioms hold: 


(i) R(C)=C for a constant C. 

(ii) R(AX) = AR(X) when A > 0 (positive homogeneity). 
(iii) R(X +X") < R(X) + R(X’) (subadditivity). 
(iv) R(X) < R(X’) when X < X’ (monotonicity). 


We here assume that the random variable X, to which 7(-) is applied is ori- 
ented such that large values are undesirable. In SP the random variable is F(x, V). If 
R(X) > E[X], for a nonconstant random variable X, then R(-) is averse. Obviously, 
the expectation is not averse. 

Based on Rockafellar and Uryasev (2000), we focus on the a-superquantile risk 
measure 


R(X) = Ga(X), (11.6) 


which is coherent and averse for a > 0, where q,(X) is the a-superquantile of X 
defined by 
Ga(X) = E[X | X > qo(X)], (11.7) 
with q,(X) being the a-quantile of X, and a € [0,1] being a probability level. We 
note that for a probability level of a = 0, we obtain that E[X] = qo(X), and for a 
probability level of a = 1, we get sup X = q@(X), ie., it is equivalent to analyzing 
the worst-case scenario. We let SP-E denote SP with R(-) = E]-] and SP-S, denote 
SP with R(-) = Ga(-). 
By Rockafellar and Uryasev (2000), 
da(F(x2,V)) = min z+ ——Blmax{ F(x, V) — z, O}]. (11.8) 


Hence SP-S, involves optimizing over x and the auxiliary variable z, and the objective 


function 
1 
z+ ear [max{F'(x,V) — z,0}). (11.9) 
—a 


SP-S, can be reformulated as a large-scale convex smooth nonlinear program (Rock- 
afellar & Uryasev, 2000). However, we implement SP-S, with the exponential smooth- 
ing technique (Kort & Bertsekas, 1972) and obtain the following approximate objec- 
tive function 

1 


Zz eer hE Ge ea (11.10) 


where 
1 
G,(z, z,v) = eee {p (F(z, v) — z)} +1) (II.11) 
and p > 0, which is a smoothing parameter. G,(-,-,v) is a continuously differentiable 


function for all v. With some mathematical manipulations, we can see that 


In 2 
0< G,(z,z,v) — G(z,z,v) < = (11.12) 
for all x, z,v, and p > 0, where 
G(a, z,v) = max{F(z,v) — z, 0}, (11.13) 


see, e.g., Pee and Royset (2011). 
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D. SCENARIO AND TARGET PATHS 


We consider an AOI represented by a grid of eleven by eleven cells, numbered 


from right to left, from top to bottom, as shown in Figure 1. 





Figure 1. AOI represented by a grid of 11 by 11 cells. 


The searcher initial position is cell 61. According to information gathered 
via intelligence describing the target position in cell 66 ten time periods prior to the 
mission start, we define three distributions for the target initial location as shown in 
Figures 3, 4, and 5. We obtain these target initial distributions using a Markov chain 
target model for different stationary probabilities, denoted by p, which represent the 
probability of the target staying in the same cell for the following time period. Figure 


2 represents the allowable movements denoted by S(c). The sum of the probabilities 


0.15 
a) * b) = 
0.15, Gj4_.0.15 0.2, Of4_,0.2 
{ 1 
0.15 0.2 





Figure 2. Example of target set of allowable movements for p = 0.4: a) in the middle 
of the AOI; b) at the boundary of the AOI; c) at the corner of the AOI. 
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of the target moving to the adjacent cells is 1 — p, and all these allowable cells have 
equally likely probabilities. 

We consider three different target stationary probabilities p and obtain three 
corresponding initial target distributions, the first for p = 0.6, the second for p = 0.4, 
and the last for p = 0.2; see Figures 3, 4, and 5, respectively. The numbers in the 





Figure 3. Searcher (S) initial position and target initial distribution for p = 0.6. 


cells on Figures 3, 4, and 5, correspond to the probability of the target being in that 
particular cell at time period 1, given the target stationary probability p. The more 
red the cell is, the higher the probability of the target being in that cell. A larger p 
implies a faster target. 

For this thesis, we randomly generate 100,000 independent target paths from 
the Markov chain induced by p and S(c), for the three distinct values of p. We here 
use a Markov chain for simplicity in implementation. Any target motion model can 


be used. In reality these paths could arrive via intelligence prior to the mission start, 


12 





Figure 5. Searcher (S) initial position and target initial distribution for p = 0.2. 
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e.g., data acquired by AIS (Automatic Identification System), a tracking system used 


on board ships. 


E. CASES 

In the next two chapters, we consider three different cases of SP. In Case A 
we assume that the detection rates (R.t)cecrey are deterministic, where in Case B 
the detection rates are random with a known probability distribution. We discuss 
Cases A and B in detail in Chapter III. In Case C, the probability distribution of the 
detection rate is unknown. We use regression techniques to approximate the detection 


rate by a linear combination of well-known factors. We consider Case C in Chapter 


IV. 
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III. SEARCH PLANS UNDER 
DETERMINISTIC AND RANDOM 
DETECTION RATE 


This chapter considers Cases A and B, as described in Section II.E. We use 
the expectation and the superquantile as the risk measures of the random objective 


function, and we compare their corresponding results. 


A. DETERMINISTIC DETECTION RATE 
In this section, we are interested in situations where the random detection rate 
(Re+t)cecteT takes on deterministic values. We use the expectation and superquantile, 
models SP-E and SP-Sg, respectively, as explained in Section II.C. We first consider 
the situation where the detection rate is constant over all cells and time periods. 
Using the information contained in USCG (2009) as a reference, and for il- 
lustration purposes we define visibility conditions as poor, fair, or good and the 


corresponding visual ranges in nautical miles (nm) are given in Table 1. In poor, fair, 














Visibility Visual Range Deterministic 
(nm) Detection Rate r.4 

Poor i 1.1824 

Fair 5 1.8237 

Good 10 2.8715 




















Table 1. Deterministic detection rate (R.+)-ecrer for poor, fair and good visibility. 


and good visibility, we set the probability of detecting a target in a cell during one 
time period, given that the target and a searcher are present, to values that when 
used in the numerical examples return optimal objective function values. We refer 
to this probability as the glimpse detection probability g.,. The detection rate r., 
then follows using the relationship r.4 = —In(1 — g-z); see Royset and Sato (2010). 
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Table 1 gives the resulting detection rates. These chosen detection rates allow us to 
compare results from this chapter with results obtained in Chapter IV, but are not 
representative of real search sensor performances. 

We are also interested in situations where the detection rate (R.+)cec ter is de- 
terministic, but varies between cells and time periods. So we construct three problem 
instances with such detection rates by independently sampling from three lognormal 
distributions with parameters jz = In(m;) — 07/2, where m,, i = 1, 2,3, are the detec- 
tion rates of Table 1 and o = 0.1 such that the lognormal distributions have means 
corresponding to those in Table 1. We independently generate one detection rate per 


cell and time period and we refer to Table 2 for the sample statistics summary. 

















Visibility Sample Mean | Sample Variance 
Poor 1.1846 0.0138 
Fair 1.8259 0.0335 
Good 2.8784 0.0829 




















Table 2. Summary of the sampled detection rates for the given visibility conditions. 


This results in 1,210 sampled detection rates, with smoothed density plots given in 


Figure 6. 


B. RANDOM DETECTION RATE WITH KNOWN DIS- 
TRIBUTION 


We now discuss Case B where the detection rate (R.+)-ecte7 is random with 
a known probability distribution. The detection rate is constant for all cells and time 
periods, but that constant is random. 

For Case B, we also consider the same three visibility conditions, and we 
construct three random variables with lognormal distributions having the same pa- 
rameters as for Case A, describing the detection rate for the three different visibility 


conditions, in order to compare the results of Cases A and B. Here we generate 10 
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Density 








— Poor Visibility 
—— Fair Visibility 
—— Good Visibility 











Figure 6. Sampled detection rate density plots for poor, fair, and good visibility. 


lognormal detection rates per visibility condition; see Table 3 for the sample statistics 


summary. 


1.5 2.0 2.5 


3.0 3.5 4.0 


Detection rate 





























Visibility Sample Mean | Sample Variance 
Poor 1.2261 0.0128 
Fair 1.7281 0.0334 
Good 2.9425 0.1267 





Table 3. Summary of the 10 sampled detection rates for poor, fair, and good visibility. 


During the next section we discuss and analyze the results obtained for Cases 


A and B, considering different risk measures and probability levels. 


C. NUMERICAL RESULTS 


We implement SP in the programming language General Algebraic Model- 
ing System (GAMS) on a personal computer with 8.00 GB of RAM and 2.80 GHz 


alg 








processor running Windows 7. SP is solved using the MINOS solver with default 
options. 

We let TJ = 10, and fix p = 500 for the exponential smoothing technique 
described in Section II.C. The corresponding smoothing error is bounded by 0.001386, 


which is insignificant in our context. 


1. Case A - Deterministic Detection Rate 

Table 4 shows the obtained results for Case A where we have a constant detec- 
tion rate equal for all cells and time periods, for various target stationary probabilities 
p, visibility conditions, and probability levels a. Although we do not show the results 
for a = 0, the corresponding probabilities of missing the target are exactly the same 


as the ones obtained using SP-E. 












































SP-S, 
p | Visibility | SP-E | a=0.25 | a=0.50 | a=0.75 | a=0.90 
0.6 | Poor 0.6517 0.7399 0.8016 0.8577 0.8912 
0.6 | Fair 0.5492 0.6402 0.7161 0.7903 0.8369 
0.6 | Good 0.4303 0.5171 0.6018 0.6939 0.7564 
0.4 | Poor O.¢212 0.8059 0.8557 0.8940 0.9142 
0.4 | Fair 0.6302 0.7241 0.7884 0.8413 0.8699 
0.4 | Good 0.5197 0.6152 0.6932 0.7634 0.8032 
0.2 | Poor 0.7619 0.8406 0.8813 0.9101 0.9245 
0.2 | Fair 0.6788 0.7693 0.8238 0.8646 0.8848 
0.2 | Good 0.5742 0.6710 0.7401 0.7962 0.8244 
































Table 4. Case A - Optimal objective function values for models SP-E and SP-S,, 
with a = 0.25, 0.50, 0.75 and 0.90, for a constant detection rate equal for all cells 
and time periods, where the objective function is the risk measure of the probability 
of missing the target. 


From Table 4 we observe that the better the visibility conditions the smaller 


the probabilities of missing the target, as we would expect. For larger values of p, 
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we obtain the same result. Thus for better visibility and a slower moving target, 
the searchers have a better chance of detecting the target. We also observe that 
the difference between the objective function values for different probability levels 
a decreases with the stationary probability p, e.g., for poor visibility, the difference 
between the results obtained for a = 0.75, and a = 0.90 is smaller for p = 0.2 than 


for p = 0.6 (0.8912 — 0.8577 > 0.9245 — 0.9101). 
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Figure 7. Case A - Cumulative distribution functions for the probability of missing the 
target given optimal search plan x and deterministic detection rate (good visibility, 
p= 05), 


In Figures 7, 8, and 9, we compare the cumulative distribution functions for 
F'(x,V) for the corresponding optimal search plans, when the visibility condition is 
good, with p = 0.6, p = 0.4, and p = 0.2, respectively. There is a notable distinction 
between using expectation and the superquantile with different probability levels a. 


If the decision maker wants to make sure that the probability of missing the target 


does not exceed a certain threshold, then the best approach is to rely on the outcome 
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Figure 8. Case A - Cumulative distribution functions for the probability of missing the 
target given optimal search plan x and deterministic detection rate (good visibility, 
p= UA), 


of the model SP-S, for higher probability levels a. Using Figure 7 as an example, if 
the goal is to prevent getting probabilities of missing the target of 70% or higher, then 
the decision maker should rely on the superquantile with probability level a = 0.90, 
among the four risk measures that we present. With that choice there is a 92% 
chance of obtaining better mission outcomes than the goal, as indicated by the blue 
horizontal dashed line in Figure 7, while there is only an 81% chance of doing better 
than the goal when using the expectation; see the red dashed line. This difference 
might be critical when choosing a search plan. As we see in Figures 8 and 9, this 
difference is even more significant for smaller values of the stationary probability p, 
i.e., the faster the target moves, the more beneficial in some sense it is to use the 
model SP-S, with higher probability levels a. 

We also consider the situation where the detection rates vary between cells 


and time periods, and we obtain the results of Table 5, for distinct target stationary 
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Figure 9. Case A - Cumulative distribution functions for the probability of missing the 
target given optimal search plan x and deterministic detection rate (good visibility, 
p=02) 


probabilities p, visibility conditions, and probability levels a. 

The fact that almost all the obtained results in Table 5 are smaller than the 
ones from Table 4 is due to the fact that the detection rate changes over the AOI and 
time and allows the searcher to choose cells that for a given time period have better 
detection rates. Therefore we obtain smaller objective function values. With this 
example, we note the importance of good intelligence on visibility conditions since we 
obtain more accurate detection rates and consequently more precise objective function 
values. 

Besides the probabilities of missing the target, it is useful to compare the 
corresponding search plans. Figures 10 and 11 illustrate the optimal search plans 
in Case A for good visibility conditions and target stationary probability p = 0.6, 
using SP-E and SP-So.75, respectively. The horizontal axes show the cells in the AOI 


displayed by number and the time periods considered during the mission. The vertical 
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SP-Sq 
p | Visibility | SP-E | a=0.25 | a=0.50 | a=0.75 | a=0.90 
0.6 | Poor 0.6451 0.7346 0.8004 0.8563 0.8904 
0.6 | Fair 0.5427 0.6363 0.7149 0.7913 0.8385 
0.6 | Good 0.4226 0.5122 0.6013 0.6947 0.7584 
0.4 | Poor 0.7159 0.8037 0.8543 0.8932 0.9135 
0.4 | Fair 0.6239 0.7214 0.7894 0.8430 0.8715 
0.4 | Good 0.5124 0.6127 0.6928 0.7646 0.8054 
0.2 | Poor 0.7572 0.8387 0.8803 0.9095 0.9242 
0.2 | Fair 0.6734 0.7686 0.8256 0.8661 0.8860 
0.2 | Good 0.5679 0.6687 0.7401 0.7974 0.8263 
































Table 5. Case A - Optimal objective function values for models SP-E and SP-S,, 
with a = 0.25, 0.50, 0.75 and 0.90, for a detection rate varying between cells and 
time periods, where the objective function is the risk measure of the probability of 
missing the target. 


axis is the fraction of searchers in cell c at time period t. The color red turning into 
black represents the time evolution. 

Comparing Figures 10 and 11, we realize that the model SP-So.75 spreads the 
searchers over the AOI, covering a larger area with smaller searcher fractions. Hence, 


the plan more effectively handles exceptional target paths. 


2: Case B - Random Detection Rate With Known Dis- 
tribution 


Table 6 shows the obtained results for Case B where we have a constant de- 
tection rate equal for all cells and time periods, whose value is random, for various 
target stationary probabilities p, and probability levels a, when we have good visibil- 
ity. Since the numerical results are similar to the ones of Case A, we do not present 
the results for poor and fair visibility. From Table 6 we observe that the obtained 


optimal objective function values increase with the value of a, similar to Case A. 
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Figure 10. Case A - Optimal search plan for the model SP-E, with p = 0.6, for a 
detection rate constant for all cells and time periods. 
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Figure 11. Case A - Optimal search plan for the model SP - S, = 0.75, with p = 0.6, 
for a detection rate constant for all cells and time periods. 
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SP-Sq 
|p | Visibility || SP-E | a=0.25 | a=0.50 | a=0.75 | a=0.90 











0.6 0.4142 0.5027 0.5906 0.6887 0.7602 
0.4 | Good 0.5041 0.6013 0.6830 0.7539 0.8131 
0.2 0.5593 0.6579 0.7309 0.7949 0.8399 











Table 6. Case B - Optimal objective function values for models SP-E and SP-S,, 
with a = 0.25, 0.50, 0.75 and 0.90, for a detection rate constant over all cells and 
time periods, where the objective function is the risk measure of the probability of 
missing the target. 


In Figure 12 we compare the cumulative distribution functions of F(z,V) for the 
corresponding optimal search plans for good visibility conditions, with p = 0.4. We 


notice a clear distinction between the four risk measures. In this example, the goal 
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Figure 12. Case B - Cumulative distribution functions for the probability of missing 
the target given optimal search plan x and random detection rate constant over all 
cells and time periods (good visibility, p = 0.4). 
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is to establish a threshold for F'(x,V) and consider the worst 20% of scenarios. In 
Figure 12, a horizontal dashed line marks the 80% of scenarios. The vertical dashed 
lines mark the corresponding thresholds for the four risk measures. The horizontal 
dashed line crosses each risk measure cumulative distribution function and gives cor- 
responding probabilities of missing the target on the horizontal axis, as indicated by 
the colored vertical dashed line. In this particular example, we note that the worst 
20% of scenarios involve a probability of missing the target greater than 84% for the 
expectation, where as only 65% for the superquantile with a = 0.50. Hence, the 
decision maker should rely on the superquantile with a = 0.50 and therefore avoid 


higher probabilities of missing the target. 
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Figure 13. Case B - Cumulative distribution functions for the probability of missing 


the target given optimal search plan x and random detection rate constant over all 
cells and time periods (good visibility, p = 0.6). 


Figure 13 shows a final example for Case B. In this situation we assume that the 


decision maker is concerned with what risk measure to use in the objective function 
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for a given threshold. Once the threshold is established, we note which cumulative 
distribution function has larger value. Here we check which interval between vertical 
dashed lines the threshold is in, and the color of the smallest of the two lines indicates 
which risk measure the decision maker should rely on. For example, if the threshold 
is 65%, then the decision maker should rely on the superquantile with a = 0.75, as 
shown by the green dashed line. Similarly, if the threshold is 80%, then the probability 
level should be a = 0.90. We note that we should rely on expectation only if the 
threshold is smaller than 46% for the probability of missing the target. 
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IV. SEARCH PLANS UNDER UNKNOWN 
DETECTION RATE 


This chapter considers Case C, as described in Section I.E. We assume that 
the probability distribution of the detection rate is unknown, however the decision 
maker has a table of realizations available. 

The detection rate performance of the sensor equipped on board a searcher 
is usually complicated to determine. We assume that there exist some factors that 
describe the detection rate in some sense and whose probability distributions we know 
and are easy to handle. These factors could be environmental conditions and target 
characteristics. We next discuss how to approximate the detection rates using these 


factors. 


A. APPROXIMATION OF RANDOM DETECTION RATE 


Suppose that we adopt the approximation 


Re Hee = Reg oh ot) = a3 BLY, cec,teT, (IV.1) 


2 2 


where Y is a vector of factors with known joint distribution and (2, and 6,4 are re- 


gression coefficients. Then the approximate probability of missing the target becomes 


EG, V) a F(a, V) = exp {- Ss" Pet (Boe + fv re.) cs (IV.2) 


cc! ,teT 
Since we approximate the random variable (Re+)cecrzey by a linear combination of 
different factors, we need to ensure that the difference between the risk measure 
of the true random objective function and the risk measure of the approximated 
one, R(F(x,V)) — R(F(x,V)), is bounded in some sense, giving us a reasonable 


approximation to use in SP. The following theorem proves such bounds. 


Zi 


Theorem IV.1. For a risk measure R(-) that satisfies the axioms stated in section 


IT.C, if we have Rey > 0 almost surely for allc EC andt € 7, then for any x > 0 


— S- Lockth: (max {0, lee Res }) 


cc! ,teT 


R(F(x,V)) — R(F(2,V)) 
S- Lewahe (max {0, Rey — Res }) ; (IV.3) 


cc! ,teT 


IA 


IA 


Proof. 
For any x > 0 and realizations v = ((pet; Pet)! )cecter and 6 = ((Det, Pet) Jeceser of 


V and V, respectively, we have that 


F(a, v)—F (2,6) = exp {- ye parte) —exp {- S° pafstoes} . (IV.4) 


cc! teT cc! ,teT 


Thus, 


|F'(z,v) = se) = 


y Pet ctXe,c t y Pet’ ctXe,c! t 


cc! ,tEeT cc! teT 


(IV.5) 








where we use the fact that the exponential function is Lipschitz continuous with con- 
stant 1, provided that we are dealing almost surely with non-positive exponents. Since 
Pot 2 O and p.z > 0, for the cases where >), o pep PethetTeet 2 Dect ter Pet et Lee! ts 
we obtain that 


F(z,v) =: F(a, v) = €xp {- S- parton] — €xp {- oa pater) 


cc! ,teT cc! ,teT 


s S- Deilag lee — S- PetXe,c! t1 ct: (IV.6) 


cc! ,teT cc! ,teT 


On the other hand, for the cases where Deeier Pel Gipsy & wie Del oibeds 


we get 


Ea) =< F(z, 0) = €xp {- S- paraater| — exp {- S- pater) < 0. 


c,c! tEeT cc! ,tEeT 
(IV.7) 
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Hence, for any 7, > 0, combining both inequalities (IV.6) and (IV.7), and since 


Pcyt = 0 
Be, v) = F(a, v) = exp {- Se porate) — €xp {- be paar) 
c,c! tEeT cc! ,tEeT 
< max ‘° S> Dotto ther- >, potcerral 
c,c! tEeT cc! ,teT 
= Max ‘° y Dowtodill et = ra} 
cc! ,tEeT 
< S Pct%e,c! t Max {0, Fe > rer} 
cc! ,tEeT 
< Ss" Lect max {0,7ot —Tet}. (IV.8) 
cc! ,tEeT 
We therefore obtain that 
F(a,v) < F(a,6)+ »- Lect max {0, Foz — Tet}. (IV.9) 


cc! teT 
The result holds if we take the risk measures on both sides, based on the 


axioms stated in section II.C, 


R(F(2z,V)) < R (Fe V) + S- x opt {0, Ret — ra} 


cc! ,tEeT 


< R(F(a, V)) +R ( S- Lec! t MAX {0, ie = ra} 


cc! ,teT 





< R(F(2,V))+ Ss" LeetR (max {0, ie — Res }) . (IV.10) 


cc! ,teT 
And we obtain the upper bound 
R(F(2,V)) —R(F(#,V)) < S> aeeeR (max {0, Res — Rep }), (IV.11) 
cc! ,teT 
In order to obtain a lower bound, we use the same approach as for the up- 


per bound. Now for the cases where par Wise ele be eer DetVct£c,ct, We 


29 


obtain the inequality 


Eg, v) _ Dees v) = €xp {- S- patter] — exp {- S- parater| < 0. 
c,c! tEeT cc! ,tEeT 
(IV.12) 


And whenever ee teT Pethetbee x ee teT Petlot bee tr we get 


exp _ y pafeator| — exp _ y parton 


cc! ,teT cc! ,teT 


S- Protec! tT ct — S- Dotto sTovs (IV.13) 


cc! ,teT cc! ,teT 


F(az,6) — F(a, v) 


IA 


Hence, for any 7.4 > 0, combining both inequalities (IV.12) and (IV.13), and since 
Pc,t 2 0 


BGO) Pae) Ss max {0 SS" Pet®eet Per — >, petaeiel 
c,c tEeT c,c’ tEeT 
< S- Lee! t max {0, rot — Tet} - (IV.14) 


cc! ,teT 
After applying the same axioms as for the upper bound calculation, in (I[V.10), we 


get the following lower bound 


R(F(2,V)) — R(F(2,V)) >— S> weoaR (max {0, Reg = Rea} ) _ (IV.15) 


cc! ,teT 











So combining both results stated before, we obtain the result. 





We use the result of Theorem IV.1 in the next section. 


B. RISK-TUNED REGRESSION MODELS 

We would like to compute the regression coefficients 3 = (Bee Cow) deeqer Ot 
(IV.1) in such a way that we minimize the difference R(F(x,V)) — R(F(x,V)). In 
view of Theorem IV.1, we minimize a weighted sum of the lower and upper bounds as a 
surrogate of that difference. The weights w , w2 € [0,1] are determined by the decision 


maker in a suitable manner. As an example on how to choose these weights, we have 
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that large probabilities may be more unfavorable in SP, therefore the upper bound 
should be more weighted. Overestimating the detection rate gives larger objective 
function values in SP, but in reality it translates into higher probabilities of missing 
the target. 

Consequently we obtain the following optimization problem for determining 


the regression coefficients: 


Model RP(wy, w2): 


min fe S- R (max {0, Ree = Ree(B)}) + We yy R (max {0, Fe e(B) - ra})| 


c,c! tEeT cc! ,tEeT 
(IV.16) 
s.t. ae Bae 20 VOLE T Gs: (IV.17) 


Constraint (IV.17) establishes that the approximated detection rate must be 
nonnegative, almost surely, for every cell and time period, since the true detection rate 
never takes on negative values. We note that RP(w , w2) is a nonstandard regression 
problem. 

We assume that we have a table of N realizations of the detection rates and the 
corresponding realizations of the factors. Let ea ee be these realizations, where 
y* and r* correspond to the k” realization of the factors Y and the detection rate 
R, respectively. For simplicity, we here assume that R,, is identical in all cells and 
all time periods. Our methodology applies also beyond this assumption. However, 
it then would require more data and calculations. In situations where the table of 
realizations depends on cell c and time period t, we have a total of CT auxiliary linear 
optimization programs to solve. 

We use superquantile as the risk measure in RP(w, w2), and with the discrete 
data mentioned above, we transcribe RP(w, w2) into a linear program and compute 


the regression coefficients 8 = (8°, 8)", where we drop the subscripts ¢ and t since all 


dl 


cells and time periods are identical. We then use the obtained regression coefficients 
in the objective function of SP by replacing the true random vector (Ret)cec ter by 


the approximated detection rate Ret — Roi(P°, By, as seen in the following section. 


C. SEARCH PROBLEM UNDER UNKNOWN DETEC- 
TION RATE 


In SP, after obtaining the regression coefficients, we substitute the true objec- 


tive function 


F(az,v) = exp {- S- parte) 


cc! tEeT 


by the following approximation 


F(a,0) = exp {- S- Des(B° + PT itces} : (IV.18) 


cc ,teT 


and we obtain a new model of SP as follows: 


Model SP: 


min R Gr {- Dy Pes(B° ale PT arces} (IV.19) 


cc! tEeT 
st. (IL3) - (IL5) 


In SP we use superquantile with a given probability level a as the risk measure. 
The value of a is the same one used in RP(wy, we). 

In order to further simplify the computational effort for the next section, we 
only take into account one factor: visibility conditions in the AOI, measured in nau- 
tical miles, during the mission’s time horizon. We consider situations where we might 
not have predictions of the visibility conditions for the mission time horizon, for ex- 
ample situations where the mission needs to be planned ahead in time and we do not 
know which visibility conditions to expect. We utilize the N realizations ae reyy 


in RP(w,,we). Then we use the distinct realizations {yen in SP to estimate a 
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probability distribution of Y, which we assume is independent of the probability 
distribution over the target paths, and then apply these distributions in SP. 


In our numerical example, we assume that we have N = 30 realizations, where 


yi =...=y'° =1, and r,...,r1° are randomly generated using a lognormal distribu- 
tion, yt =... = y?9 = 5, and r""...,r?° are randomly generated using a lognormal 
distribution, and y”4 = ... = y®° = 10 nautical miles, and r?',...,r°° are randomly 


generated using a lognormal distribution. All the lognormal distributions have the 
same parameters as for Case A, according to the visibility conditions. After obtaining 
the regression coefficients from the model RP(wy, w2), we use the three distinct y* in 


the model SP. 


D. NUMERICAL RESULTS 

We next present numerical results of RP(w,, w2) and we compare the regression 
coefficients obtained using weights w, = 0.25 and w2 = 0.75, for different probability 
levels a, with the traditional least-squares linear regression. We show two different 
plots, one to represent RP(0.25, 0.75), the other to represent RP(0.75, 0.25), in order 


to demonstrate that the values of the weights w , and wg are fairly important. 
































ay Axis Interception 39 | Slope 3 
0 0.9567 0.1645 
0.25 0.9567 0.1645 
0.35 0.9367 0.1665 
0.50 0.8740 0.1728 
0.75 0.8024 0.1750 
0.80 0.6378 0.1816 
0.85 0.6925 0.1707 
0.90 0.6967 0.1698 




















Table 7. Case C - Regression coefficients 8 = (8°, 8)' for different probability levels 
a, in RP(0.25, 0.75). 
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Table 7 shows the obtained regression coefficients 8 = (6°,3)' for different 
probability levels a, and weights w; = 0.25, w2 = 0.75. In the traditional least-squares 
regression we obtain 6 = (0.9665, 0.1886)". 
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Figure 14. Case C - Comparison between least-squares linear regression and RP(0.25, 
0.75), for different a. 


Figure 14 shows the corresponding linearly approximated functions using least- 
squares linear regression and RP(0.25, 0.75). We note that the nonstandard regression 
RP(0.25, 0.75) tends to shift the fitted lines towards the smallest realizations of 
{rey as the probability level a increases, therefore underestimating the detection 
rates and returning more conservative regression coefficients. Figure 15 shows exactly 
the opposite because the weights are the complements of the ones used in Figure 14. 
This implies that the decision maker should be careful defining the weights w, and 
Wo. 

After obtaining the regression coefficients, we replace the objective function by 
the approximated one in SP, as described in Section IV.C. Table 8 presents the opti- 


mal values for Case C. We observe that Case C results in larger optimal values than 
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Figure 15. Case C - Comparison between least-squares linear regression and RP(0.75, 
0.25), for different a. 


the previous cases, since the nonstandard regression generates conservative estimates 


of the detection rate for SP. 



































SP-S, SP 
Case A Case A Case B Case C 
p | Visibility | (R.4 constant (R.4 varies (random R., | (random R.4 
Ye, t) between c and t) | constant Vc,t) | constant Vc, t) 
0.6 0.6939 0.6947 0.6887 0.8465 
0.4 Good 0.7634 0.7646 0.7539 0.8831 
0.2 0.7962 0.7974 0.7949 0.9007 





Table 8. Comparison of optimal objective function values between Cases, using su- 
perquantile with a = 0.75 (Good visibility, w, = 0.25, w2 = 0.75), where the objective 
function is the risk measure of the probability of missing the target. 


For the situation where the regression coefficients are obtained by using the 
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least-squares linear regression, we use a = 0 in SP. Figure 16 shows that the optimal 
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F(x,v) 


Figure 16. Case C - Cumulative distribution functions for F'(z,V) given optimal 
search plan x and random detection rate. Comparison between least-squares and 
RP(0.25, 0.75) regression coefficients (Good visibility, p = 0.6). 


search plan for the least-squares performs better for thresholds for F'(z,V) up to 
45%. Then using a = 0.50 is the best solution until a threshold of 66%. After a 66% 
probability of missing the target it is almost indistinguishable which a to choose, 
therefore one consideration should be the run times of SP for each value a. From all 
the numerical examples, we note that the larger the a, the longer the run time of SP. 
From Figure 16 we observe that if we establish a goal of preventing probabilities of 
missing the target of 70% or higher, we have 92% chance of doing better than the 
goal if we rely on RP(0.25, 0.75) with a = 0.75 to obtain the regression coefficients 
and use them in SP, while only 74% chance if we use the least-squares regression 
coefficients in SP. 

Figures 17 and 18 show the same comparison for fair and poor visibility condi- 


tions. We notice that the optimal search plans obtained using least-squares regression 
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F(x,v) 


Figure 17. Case C - Cumulative distribution functions for F'(2,V) given optimal 
search plan x and random detection rate. 
RP(0.25, 0.75) regression coefficients (Fair visibility, p = 0.6). 
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Figure 18. Case C - Cumulative distribution functions for F'(z,V) given optimal 
search plan x and random detection rate. 
RP(0.25, 0.75) regression coefficients (Poor visibility, p = 0.6). 
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Comparison between least-squares and 


perform better for smaller thresholds of F(x, V). However towards the tail of the cu- 
mulative distribution function of the probability of missing the target, using larger 


values of a gives higher chance of performing better than the established threshold. 
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Vv. CONCLUSIONS 


In this thesis, we discuss a type of stochastic optimization problems where the 
decision maker does not have complete information concerning the objective function. 
Specifically, we consider a discrete time-and-space search optimization problem (SP). 
We want to route search assets in order to detect a moving target such as a drug 
smuggler in an area of operations. 

We assume that each searcher is equipped with one imperfect sensor, which 
is not subject to false-positive detections. The searchers are all alike, equipped with 
the same type of sensor, and they are allowed to divide their effort across multiple 
cells in arbitrarily small portions. The goal is to determine a search plan such that 
the probability of missing the target is minimized, without having the full knowledge 
of the two sources of uncertainty: the target location and the sensor performance. 
We assume that the target paths are probabilistically known via intelligence reports; 
for example from data acquired by AIS (Automatic Identification System). We use 
a Markov chain model to generate the paths with three stationary probabilities. We 
assume that the sensor performance depends on environmental conditions, and we 
use visibility to represent these conditions. 

We formulate the objective function of SP in terms of a risk measure of a 
parameterized random variable and consider three cases involving various degrees of 
knowledge about the sensor performance. In all cases, we consider both the expecta- 
tion and superquantile risk measures. While the expectation results in an objective 
function representing the probability of missing the target, the superquantile gives 
rise to more conservative search plans that perform reasonably well even under ex- 
ceptional circumstances. 

In the first two cases, we consider a random detection rate with known prob- 
ability distribution. For Case A we present two possible situations, the first for a 


deterministic and constant detection rate over all cells and time periods, and the sec- 
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ond for a deterministic detection rate that varies between cells and time periods. The 
numerical examples show that the objective function values for a varying detection 
rate are smaller than for a constant one, since the search includes the cells with higher 
detection rates. One example shows that if the goal is to prevent getting probabili- 
ties of missing the target of 70% or higher, then the decision maker should use the 
superquantile risk measure with a = 0.90. In that situation there is a 92% chance 
of obtaining better mission outcomes than the goal, while there is only 81% chance 
of getting better than the goal when using the expectation. This difference might be 
critical when choosing a search plan and is more significant for faster moving targets. 

For Case B the detection rate is random but constant for all cells and time 
periods. The results obtained for Case B are close to the results from Case A. We show 
different comparisons between cumulative distribution functions that demonstrate 
how a decision maker may benefit from different risk measures depending on the 
threshold for the probability of missing the target. 

In the third case, the distribution of the sensor detection rate is unknown 
and we approximate the detection rate by a linear combination of well-known factors 
using a nonstandard regression. We use visibility conditions as the factor in the nu- 
merical examples. Based on a table of observed sensor performances, that regression 
minimizes the error induced by the approximation in some sense. We compare the 
obtained coefficients with the ones resulting from a least-squares linear regression 
model. We show that the coefficients obtained from the nonstandard regression are 
more conservative than the ones from least-squares, by underestimating the detection 
rate. Using the approximations in the objective function of SP, we obtain an approx- 
imate problem that we solve for situations involving stochastic information about the 
visibility conditions. The nonstandard regression utilizes all observed sensor perfor- 
mances and we obtain worse results than the ones from Cases A and B, since there 
is only probabilistic information about what visibility conditions to expect during 


the mission. However we show that the optimal search plan performs better in ex- 
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ceptional circumstances if obtained by using the nonstandard regression coefficients 
instead of the ones from least-squares linear regression in the approximate problem. 
We show that for good visibility we have an 18% higher chance of preventing proba- 
bilities of missing the target of 70% or higher if we use the search plan that relies on 
nonstandard regression coefficients with a = 0.75 in SP. 

In all cases, the numerical results show that the superquantile spreads the 
searchers over the area of operations, covering a larger area with smaller searcher 
fractions than when using expectation. This risk measure handles exceptional target 


paths more effectively. 


A. FUTURE RESEARCH 

Future research could generalize the bounds between the risk measure of the 
true random function and the risk measure of the approximated one beyond an expo- 
nential of a sum of linear functions. This generalization could lead to new nonstandard 


regression techniques. 
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